Please type a plus sign (♦) inside this box 



PTO/SB/05 !4/98> 

Approved for use through 09/30/2000. 0MB 0651-0032 
^ A Patent and Trademark Office: U.S DEPARTMENT OF COMMERCE 

Under the Paperwork Reduction Act of 1995. no persons are required o respond to a collection of information unless it displays a vaitd OMB control number 



UTILITY 
PATENT APPLICATION 
TRANSMITTAL 

{Only for new nonprovtsional applications under 37 C F R § 1 53(b)] 



Attorney Docket No. RCA 8 9541 



First Inventor or Application identifier Mar 



Title 



Comer, et 



Apparatus and Method tor (over) 



Express Mail Label No. 



EL533624930US 



al 



APPLICATION ELEMENTS 

See MPEP chapter 600 concerning utility patent application contents. 



ADDRESS TO: 



Assistant Commissioner for Patents 
Box Patent Application °* 



CXI 



nr"| * Fee Transmittal Form (e.g., PTO/S8/17) 
' ' (Submit an ongtnal and a duplicate for fee processing) 



2. Specification [Total Pagesl 3 4 

(preferred arrangement set forth below) I 

* Descriptive title of the Invention 

- Cross References to Related Applications 

- Statement Regarding Fed sponsored R&D 

- Reference to Microfiche Appendix 

- Background of the Invention 

- Brief Summary of the Invention 

• Brief Description of the Drawings (// filed) 

- Detailed Description 

- Claim(s) 
t t - Abstract of the Disclosure 

3. | Xj Drawing(s) (35 U.S.C. 113) {Total Sheets 



5. II Microfiche Computer Program ( Appendix /? co 
1 D<M 

6. Nucleotide and/or Amino Acid Sequence SubmisSgJ? ^ 

{if ap plicabl e, all necessary) crl^^ 

a. | I Computer Readable Copy q*" 3 Q 

b. I I Paper Copy (identical to computer copy) 

c. | I Statement verifying identity of above copies 



Oath or Declaration [Total Pages 

a. I X| Newiy executed (original or copy) 

h I I Copy from a prior application (37 OF.R. § 1.63(d)) 
°- | I (for conthiuation/drvisjonal with Box 16 completed) 

□ DELETION OF INVENTORY 
Signed statement attached deleting 
inventors) named in the prior application, 
se«37C.F.R. §§ 1 .63(d)(2) a nd 1.33(b). 



7. 

8. 

9. 
10. 
11. 



\FEES, A SMALL £NTtTY STATOtBiTtS REQUmEQ 07 CF.H.J 1JTL £XCfiPT | 
Q«E*L*>MA> 



ACCOMPANYING APPLICATION PARTS 



rz 
rz 
m 



Assignment Papers (cover sheet & document(s)) 
37 C.F.R.§3.73(b) Statement r—^n Power of 
(when there is an assignee) I I Attorney 

English Translation Document (if applicable) 
Information Disclosure i \ o\ C°P* es of IDS 



12. nr 

□ 



Statement (IDSyPTO-1449 Citations 

Preliminary Amendment 

Return Receipt Postcard (MPEP 503) 
(Should be specifically itemized) 

•Small Entity j . statement filed in prior application, 

fprSs»wt2;' ' status stilt proper and des(red 



14. 



□ Certified Copy of Priority Document(s) 
(if 



(if foreign priority is claimed) 

15. | | Other 



16. If « CONTINUING APPLICATION, check appropriate box, and supply the requisite information below and m a preliminary amendment: 

| | Continuation Q Divisional Q Cortouaton-in-part (C1P) of prior application No: / 

Prior appHcation information: Examiner Group / Art Unit 



For CONTINUATION of DfvTStONAL APPS onlv: The entire disclosure of the prior application, from which an oath or declaration la supplied 
under 8 ox 4b, is considered ■ pert of the disclosure of the accompanying continuation or divisional application and ia hereby Incorporated by 
reference. The Incorporation can only be railed upon when a portion has been Inadvertently omitted from the submitted application parts. 



□ Customer Number or Bar Code Label 



1 or I_l Correspondence address below 



Name 



Joseph S. Tripoli 



Thomson Multimedia Licensing Inc. 



Address 



P.O. Box 5312 



Two Independence Way 



City 



Princeton 



State 



NJ 



Zip Code 



08543 



Name (PrmtrType) 


Alexander J. Burke | Registration No. (Attorrwy/Agm) 


40,425 


Signature 







comments on the amount of time you are required to-complete this form should be sent to the Chief Information Officer, Patent and Trademark Office, 
Washington, DC 20231 . DO NOT SEN0 FEES OR COMPLETED FORMS TO THIS AOORESS. SEND TO: Assistant Commissioner for Patents, 
Box Patent Application, Washington, DC 20231. 



RCA 89,541 



Express Mail: EL 5 3 3 6 2 4 9 30US 



1 

APPARATUS AND METHOD FOR DERIVING AN ENHANCED DECODED 
REDUCED-RESOLUTION VIDEO SIGNAL FROM A CODED HIGH-DEFINITION 

VIDEO SIGNAL 

5 This is a non-provisional application of provisional application serial No. 

60/133,429 by M. L. Comer et al, filed 1 1 May 1999. 

Field of the Invention: 
The present invention relates to the decoding of a coded high-definition (HD) 
10 video signal to derive an enhanced decoded video signal suitable, for example, for 
recording or producing a picture-in-picture (PIP) or other reduced-resolution display. 

Description of the Prior Art: 
Known in the art are television receivers that, while displaying a relatively large 
1 5 picture derived from a primary television channel, also simultaneously display a small 
picture-in-picture (PIP) derived from a secondary television channel. In the case of a 
high-definition television (HDTV) receiver, the receiver must include a relatively 
complex and expensive decoder that conforms with the MPEG ISO 13818-2 standard 
for decoding a received coded HD video signal in real time for high definition display. 
20 However, because the PIP is small, there is no need to provide a high definition PIP 
display because a viewer inherently would not be able to resolve the higher definition 
components of a high definition PIP. Therefore, to provide the PIP, the HDTV receiver 
may be supplied with a lower-resolution second simpler and less expensive decoder 
which still conforms with the ISO 13818-2 standard. 
25 One approach, known in the art, to providing a lower-resolution second decoder 

which is somewhat simpler and less expensive than the decoder providing the high 
definition display, is disclosed in the three U.S. patents 5,614,952, 5,614,957 and 
5,635,985, which were, respectively, issued to Boyce et al. on March 25,1997, March 
25,1997 and June 3, 1997. 
30 Further, incorporated herein by reference is the teaching of copending U.S. 

patent application S.N. 09/349,865, filed July 8, 1999 and assigned to the same 
assignee as the present application, which is directed to a lower-resolution second- 
decoder approach suitable for deriving a PIP display in real time from a received coded 
HD video signal that is significantly simpler and less expensive to implement than is 
35 the second decoder disclosed by Boyce et al, but still conforms with the ISO 13818-2 
standard. 
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SUMMARY OF THE INVENTION 



A system involves decoding compressed image data including frequency 
domain coefficients defining blocks of pixel values representing an image at a first 
5 resolution to provide an image at a reduced second resolution. The system includes a 
motion-compensation-unit (MCU) processor responsive to a selected sub-set of the 
frequency domain coefficients for deriving the image of the reduced second resolution. 
The motion-compensation-unit (MCU) processor employs blocks of pixel values 
representing image data at an intermediate third resolution lower than the first 
10 resolution and higher than the reduced second resolution. 



BRIEF DESCRIPTION OF THE DRAWING 



FIGURE 1 is a functional block diagram showing a variable-length decoder 
15 (VLD) responsive to an input HD MPEG data bit-stream for providing a first selected 
MPEG data output to a PIP decoding means and a second selected MPEG data output 
to an HD decoding means; 

FIGURE la shows an 8x8 block containing the 64 DCT coefficients that are 
employed by the HD decoding means of FIGURE 1, FIGURE lb shows an 8x8 block 
20 containing the particular 10 DCT coefficients of the 64 DCT coefficients shown in 
FIGURE la that are employed by the PIP decoding means of FIGURE 1 for 
progressive- scan sequences and FIGURE 1c shows an 8x8 block containing the 
particular 10 DCT coefficients of the 64 DCT coefficients shown in FIGURE la that 
are employed by the PIP decoding means of FIGURE 1 for interlaced-scan sequences; 
25 FIGURE 2 is a simplified functional block diagram of an embodiment of the 

PIP decoding means of FIGURE 1 which incorporates features of the present invention; 

FIGURE 3 is a functional block diagram showing details of the enhanced MCU 
processing means of FIGURE 2. 

FIGURE 4 is a conceptual diagram showing the computational processing 
30 performed by the DCT-based upsample means of FIGURE 3; and 

FIGURE 5 is a conceptual diagram showing the computational processing 
performed by the DCT-based downsample means of FIGURE 3. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

35 



Referring to FIGURE 1, there is shown VLD 100, PIP decoding means 102 and 
HD decoding means 104. In accordance with the known teaching of the MPEG ISO 
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13818-2 standard, one of the responses of VLD 100 to the input coded HD MPEG data 
comprising a sequence of MPEG I, P and B frames is to convey coded picture 
information defined by each of successive 8x8 blocks of quantized discrete cosine 
transform (DCT) coefficients as an input to HD decoding means 104. Further, in 
accordance with the known teaching of the MPEG ISO 13818-2 standard, among the 
functions performed by HD decoding means 104 is to first perform inverse quantization 
of each successive 8x8 block of DCT coefficients and then perform inverse discrete 
cosine transformation (IDCT) of the DCT coefficients of each successive 8x8 block. 
Finally, HD decoding means 104 must perform motion compensation for each P frame 
and bi-directionally predictive B frame after IDCT has been performed on that P or B 
frame. 

FIGURE la shows an 8x8 block of DCT coefficients, wherein (1) the value of 
coefficient DCT 0 ,o (located in the upper left corner of the 8x8 block) represents the 
average (DC) value (i.e., both the horizontal and vertical frequencies are 0) of the 
picture defined by the 64 values of a corresponding 8x8 block of pixels prior to having 
undergone DCT, while (2) the value of coefficient DCT 7 , 7 (located in the lower right 
corner of the 8x8 block) represents the highest horizontal frequency and highest 
vertical frequency components of the picture defined by the 64 values of a 
corresponding 8x8 block of pixels prior to having undergone DCT. For the case of a 
HD picture, all, or nearly all, of the 64 DCT coefficients from DCT 0 ,o to DCT7/7 
inclusive of FIGURE la may have non-zero values. This results in a relatively large 
amount of image-processing computation to accomplish IDCT in real time. Further, 
motion compensation also involves a large amount of real time image-processing 
computation. Therefore, HD decoding means 104 requires about 96 Mbits memory to 
temporarily store MPEG decoded image frames prior to display. HD decoding means 
104 requires these frames for motion compensation to reconstruct accurate images for 
display. Thus, a physical implementation of HD decoding means 104 is relatively 
expensive. 

Returning to FIGURE 1, another of the responses of VLD 100 to the input 
coded HD MPEG data is to convey only coded picture information defined by a 
relatively small given number of lower-frequency-defming, quantized DCT coefficients 
of each successive 8x8 block as an input to PIP decoding means 102. It is to be noted 
that the PIP processing and images and the term PIP itself is used herein to encompass 
any form of reduced resolution image and processing and not just television PIP image 
generation. While the preferred tutorial example of the PIP decoding means described 
in the aforesaid patent application S.N 09/349,865 employed only the 6 lowest- 
frequency quantized DCT coefficients, the preferred tutorial example of enhanced- 
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quality PIP decoding means 102, described in detail below, employs 10 DCT 
coefficients consisting of DCT 0 ,o, DCT ls0 , DCT 2 ,o, DCT 3 ,o, DCTo,i, DCT U , DCT 2a , 
DCT 0 ,2, DCTi j2 , and DCT 0 , 3 shown in FIGURE lb for progressive-scan use or, 
alternatively, consisting of DCT 0 ,o, DCTi, 0 , DCT 2 ,o, DCT 0 ,i, DCT U , DCT 0)2 , DCT 0 , 3 , 
DCT 0)4 , DCT 0 ,5, and DCT 0 ,6 shown in FIGURE lc for interlaced-scan use, thereby 
providing better high-frequency response for the enhanced PIP display. More 
specifically, the PIP bit-stream received by VLD 100 has been pre-parsed by a 
VLD-PIP parser (not shown to simplify Figure 1) to remove from the bitstream DCT 
coefficients which are not needed by the PIP decoder. 

The simplified functional block diagram of the embodiment of enhanced- 
quality PIP decoding means 102 shown in FIGURE 2 comprises runlength decoder 
(RLD) 200, inverse quantizer (IQ) 202, unitary enhanced IDCT, filtering and pixel- 
decimation processing means 204, base-layer adder 205B, enhancement-layer adder 
205E, base and enhancement-layer decimated pixel memory 206, enhancement-layer 
encoder 207, enhanced motion compensation unit (MCU) processing means 208 and 
sample-rate converter 210. Although the simplified functional block diagram of 
FIGURE 2 does not show means for controlling the operation of this embodiment of 
enhanced-quality PIP decoding means 102, it should be understood that suitable control 
means that conform with the requirements of the ISO 13818-2 standard is included in a 
physical implementation of this embodiment. 

For illustrative purposes, the following description of elements 200, 202, 204, 
205B, 205E, 206, 207, 208 and 210 assumes that each of these elements is being 
operated in accordance with the above-discussed preferred tutorial example. 

In this example, RLD 200 outputs 10 DCT coefficients for each 8x8 coded 
block using the 2 scan patterns defined in the ISO 13818-2 standard. The positioning of 
the 10 DCT coefficients within each 8x8 block is illustrated in Figure lb for 
progressive scan and in Figure lc for interlaced scan, as determined by the state of the 
progressive_sequence flag. In the Figure lb progressive sequence case, if the 
alternate_scan flag from the picture coding extension for the current picture is 0, the 10 
DCT coefficients correspond to coefficients 0,1,2,3,4,5,6,7,8,9 in 1-dimensional scan 
order, whereas if alternate_scan is 1, the 10 DCT coefficients of interest are 
coefficients 0,1,2,3,4,5,6,7,8,20 in scan order. In the Figure lc interlaced sequence 
case, if the alternate__scan flag from the picture coding extension for the current picture 
is 0, the 10 DCT coefficients correspond to coefficients 0,1,2,3,4,5,9,10,20,21 in 
1-dimensional scan order, whereas if alternate_scan is 1, the 10 DCT coefficients of 
interest are coefficients 0,1,2,3,4,5,6,10,11,12 in scan order. There are two run values 
that have a meaning in RLD 200 which is different from that described in the ISO 
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13818-2 standard, depending on the value of the alternate_scan and 
progressive_sequence flags. For progressive sequences, if alternate_scan is 0, a run 
value of 10 indicates that the coefficients needed by PIP decoder 102 are all 0 and there 
is no subsequent non-zero coefficient. Similarly, if alternate_scan is 1, a run value of 
21 indicates that the coefficients needed by decoder 102 are all 0 and there is no 
subsequent non-zero coefficient. For interlaced sequences, if alternate_scan is 0, a run 
value of 22 indicates that the coefficients needed by PIP decoder 102 are all 0 and there 
is no subsequent non-zero coefficient. Similarly, if alternate_scan is 1, a run value of 
13 indicates that the coefficients needed by decoder 102 are all 0 and there is no 
subsequent non-zero coefficient. Table 1, summarizes the meaning of run values of 10 
and 21 for the two possible values of the alternate_scan flag for progressive sequence 
and Table 2, summarizes the meaning of run values of 13 and 22 for the two possible 
values of the alternate_scan flag for interlaced sequence. All other alternate_scan/run 
value combinations encountered by RLD 200 are interpreted as described in the ISO 
13818-2 standard. 



Run 


Alternate Scan 


Interpretation in PIP RLD 


10 


0 


All DCT coefficients = 0 


10 


1 


Same as ISO 13818-2 standard 


21 


0 


Not allowed 


21 


1 


All DCT coefficients = 0 


Table 1 -Interpretation of run = 10 and run = 21 by RLD 200 
for progressive sequences 


Run 


Alternate Scan 


Interpretation in PIP RLD 


13 


0 


Same as ISO 13818-2 standard 


13 


1 


All DCT coefficients = 0 


22 


0 


All DCT coefficients = 0 


22 


1 


Not allowed 



Table 2-Interpretation of run - 13 and run = 22 by RLD 200 
for interlaced sequences 



IQ 202 performs inverse quantization arithmetic and saturation described in the 
ISO 13818-2 standard on the 10 DCT coefficients shown in FIGURE lb for 
progressive sequences and shown in FIGURE lc for interlaced sequences. The 
mismatch control portion of the inverse quantization process is not needed. 
Conventionally, an extensive computational process requiring three separate steps is 
needed to convert the coded frequency domain information in an 8x8 block at the 
output from IQ 202, into spatial domain picture information comprising respective 
values of a smaller block of decimated pixels of a reduced-resolution PIP display 
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image. The first step is to determine the value of each of the 64 (i.e., full pixel density) 
pixel values of each 8x8 block of picture information as an IDCT function of the 
inversely-quantized DCT coefficient values. Thereafter, the second step of lowpass 
filtering followed by the third step of pixel decimation may be performed on the pixels 
in each successive 8x8 block to provide the desired smaller block of decimated pixels. 
For instance, a decimation of alternate horizontal and vertical filtered pixels for the 
case of a progressive scan would result in a 75% reduction in pixel density. Similarly, a 
decimation of 3 out of 4 successive filtered pixels in the horizontal direction for the 
case of an interlaced scan would also result in a 75% reduction in pixel density. Thus, 
in either case, such a decimation performed for luma pixels and also for chroma pixels 
would result in a reduction in pixel density from 64 per 8x8 block to only 16 per 8x8 
block for each of them. However, the amount of hardware required to implement this 
conventional three-step computational process is relatively large and, therefore, 
relatively expensive. 

In accordance with the principles of the invention taught in the aforesaid patent 
application S.N. 09/349,865, the unitary IDCT, filtering and pixel-decimation 
processing means disclosed therein is able to convert the coded respective values of 
inversely-quantized DCT coefficients contained in an 8x8 block at the output from IQ 
202, into a smaller block of decimated pixels in a single-step computational process. 
Thus, the amount of hardware required to implement this single-step computational 
process by means 204 is relatively small and, therefore, relatively inexpensive 
compared to the aforesaid conventional three-step computational process. 

Specifically, in accordance with the teachings of the aforesaid patent application 
S.N. 09/349,865, the decimated pixel memory thereof (which, because of pixel 
decimation, requires a storage capacity size of only 1/4 the capacity size of a 
corresponding undecimated pixel memory) comprises a plurality of separate buffers. 
Each of these buffers is capable of temporarily storing decimated luma and chroma 
pixels. In conformity with the ISO 13818-2 standard, the decimated pixel memory 
includes one or more buffers for storing decimated pixels that define reconstructed 
intracoded (I), predictive-coded (P) and/or bi-directionally predictive-coded (B) frame 
or field pictures. Further, motion-compensated prediction macroblock output of pixel 
values from the MCU processing means is added in an adder to each corresponding 
macroblock output derived in the unitary IDCT, filtering and pixel-decimation 
processing means. The summed pixel values of the output from the adder are stored 
into a first buffer of the decimated pixel memory. This first buffer may be a first-in 
first-out (FIFO) buffer in which the stored decimated pixels may be reordered between 
(1) being written into the first buffer and (2) being read out from the first buffer and 
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written into another buffer of the decimated pixel memory. In the case of a current P or 
B frame or field, the decimated pixel memory includes a buffer for storing a 
macroblock for input to the MCU processing means to provide motion compensation. 

In accordance with the principles of the present invention, two layers of 
5 decimated pixels are advantageously stored respectively in base and enhancement-layer 
decimated pixel memory 206 to achieve high-quality motion compensation and 
improve PIP image quality. The first of these two layers is a base-layer of decimated 
pixels and the second of these two layers is an enhancement-layer of vector-quantized 
values of luma macroblock decimated pixels that are employed in enhanced MCU 
10 processing means 208 during decoding of P pictures The enhanced layer is used to 
provide a reduced resolution image of greater resolution than is obtainable by using just 
the decimated pixels of the base-layer. Both the base-layer and this enhancement-layer 
are employed by enhanced MCU processing means 208, in a manner described in detail 
below. 

15 Preferred embodiments of IDCT, filtering and pixel-decimation processing 

means 204, enhancement-layer encoder 207 and enhanced MCU processing means 208 
for implementing the present invention will now be described in detail. 

The unitary enhanced IDCT, filtering and pixel-decimation processing means 
204 provides the following sets of 16 decimated pixel values for each of the base and 

20 enhancement-layers used for each of progressive scan and interlaced scan (each set 
being a function of 10 DCT coefficient values). 



PROGRESSIVE-SCAN, BASE-LAYER SET OF DECIMATED PIXEL VALUES 
g! (0,0) = [8DCT 0 ,o+ 10DCT li0 + 7DCT 2j0 + 4DCT 3 , 0 + 10DCT 0 ,i+ 13DCT U + 9DCT2,i 
25 + 7DCT a2 + 9DCT l>2 + 4DCT 0 , 3 J/ 64 

gi (1,0) = [8DCT 0 ,o+ 4DCT 1)0 - 7DCIV 9DCT 3 , 0 + 10DCT 0 ,i+ 5DCT U - 9DCT 2r i 

+ 7DCT 0 ,2+ 4DCT U + 4DCT 0 , 3 ]/ 64 
gi (2,0) - [8DCT 0> o-4DCT U o- 7DCT;> 0 + 9DCT 3 , 0 + 10DCT 0i r 5DCT U - 9DCT 2 ,i 

+ 7DCT a2 - 4DCT li2 + 4DCT 0i3 ]/ 64 
30 g, (3,0) = [8DCT 0i0 - 10DCT 1)0 + 7DCT2, 0 - 4DCT 3 , 0 + 10DCT 0 ,i- 13DCT U + 9DCT2,! 

+ 7DCT 0 , 2 - 9DCT li2 + 4DCT 0 , 3 ]/ 64 
g! (0,1) = [8DCT 0)0 + 10DCT 1)0 + 7DCT2, 0 + 4DCT 3f0 + 4DCT 0 ,i+ 5DCT U + 4DCT 2)1 

- 7DCT a2 - 9DCT U2 - 9DCTo, 3 ]/ 64 

g! (1,1) = [8DCT 0)0 + 4DCT 1>0 - 7DCIV 9DCT 3 , 0 + 4DCT 0 ,i+ 2DCTi f i- 4DCT2,i 
35 - 7DCT 0>2 - 4DCT U2 - 9DCT 0j3 ]/ 64 

gi (2,1) = [8DCT 0 ,o- 4DCT li0 - 7DCT 2 , 0 + 9DCT 3 , 0 + 4DCT 0 ,i- 2DCT U - 4DCT2,i 

- 7DCT 0j2 + 4DCT 1)2 - 9DCT 0 , 3 ]/ 64 
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g! (3,1) = [8DCT0.0- 10DCT l)0 + 7DCT2, 0 - 4DCT 3 ,o+ 4DCT 0 ,i- 5DCT U + 4DCT 2jl 

- 7DCT 0 ,2+ 9DCT ls2 - 9DCT 0 , 3 ]/ 64 

gi (0,2) = [8DCT 0 ,o+ 10DCT 1)0 + 7DCT 2 , 0 + 4DCT 3 , 0 - 4DCT 0 ,i- 5DCT U - 4DCT 2tl 

- 7DCT 0>2 - 9DCT U + 9DCT 0 , 3 ]/ 64 

5 g! (1,2) = [8DCT 0) o+ 4DCT U0 - 7DCT2, 0 - 9DCT 3 , 0 - 4DCT 0 ,i- 2DCT U + 4DCT zl 

- 7DCT 0 ,2- 4DCT li2 + 9DCT 0 , 3 ]/ 64 

g! (2,2) = [8DCT 0 ,o- 4DCT 1)0 - 7DCT 2i0 + 9DCT 3 , 0 - 4DCT 0 ,i+ 2DCT U + 4DCT2,! 

- 7DCT 0>2 + 4DCT lj2 + 9DCT 0 , 3 ]/64 

g] (3,2) = [8DCT 0)0 - 10DCT lj0 + 700X2,0- 4DCT 3 , 0 - 4DCT 0 ,i+ 5DCT U - 4DCT2J 
10 - 7DCT 0 ,2+ 9DCT lj2 + 9DCT 0 , 3 ]/ 64 

g! (0,3) = [8DCT 0i0 + 10DCT U0 + 7DCT 2)0 + 4DCT 3 , 0 - 10DCT 0 , r 13DCT u -9DCT2,i 

+ 7DCT 0)2 + 9DCT 1>2 - 4DCT 0 , 3 ]/ 64 
& (1,3) = [8DCT 0 ,o+ 4DCT U0 - 70012,0- 9DCT 3)0 -10DCT 0f i- 5DCT U1 + 9DCT2,! 
+ 7DCT 0 , 2 + 4DCT U - 4DCT 0 , 3 ]/ 64 
15 gi (2,3) = [8DCT 0 ,o- 4DCT 1>0 - 7DCT 2j0 + 9DCT 3 , 0 - 10DCT 0 ,i+ 5DCT U + 9DCT 2jl 
+ 7DCT 0 , 2 - 4DCT 1>2 - 4DCT 0j3 ]/ 64 
g! (3,3) = [8DCT 0 ,o- 10DCT U0 + 7DCIV 4DCT 3j0 - 10DCT 0 ,i+ 13DCTii- 9DCT 2ii 
+ 7DCT 0 ,2- 9DCT li2 - 4DCT 0 , 3 ]/ 64 

20 PROGRESSIVE-SCAN. ENHANCEMENT-LAYER SET OF DECIMATED PIXEL VALUES 
go (0,0) = [0DCT O)O + 0DCT u0 + ODCT 2 , 0 + 0DCT 3)0 + lDCT 0 ,i+ 1DCT U + 1DCT 2i1 

+ 3DCT 0)2 + 4DCT U + 6DCT 0 , 3 ]/ 64 
go (1,0) = [0DCT OiO + 0DCT KO + 0DCT 2 , 0 + 0DCT 3tO + 1DCT 0 ,i+ 0DCT u - IDCT^i 
+ 3DCT 0i2 + 2DCT U + 6DCT 0 , 3 ]/ 64 
25 go (2,0) = [0DCT 0)0 + 0DCT lj0 + ODCT^* ODCT 3 ,o+ 1DCT 0J + 0DCT u - lDCT^ 
+ 3DCT 0)2 - 2DCT lj2 + 6DCT 0 , 3 ]/ 64 
go (3,0) = [0DCT 0j0 + 0DCT 1>0 + 0DCT 2 , 0 + 0DCT 3 , 0 + !DCT 0 ,i- 1DCT U + IDCT^i 

+ 3DCT 0)2 - 4DCT 1)2 + 6DCT 0 , 3 ]/ 64 
go (0,2)= [0DCT 0l0 + 0DCT li0 + ODCT 2 ,o+ 0DCT 3 , 0 + 2DCT 0 ,i+ 3DCT U + 2DCT2,, 
30 + 3DCT 0l2 + 4DCT li2 - 2DCT 0 , 3 ]/ 64 

go (1,2) - [0DCT a0 + 0DCT li0 + 0DCT 2>0 + ODCT 3 , 0 + 2DCT 0 ,i+ 1DCT U - 2DCY X \ 

+ 3DCT 0i2 + 2DCT U - 2DCT 0 , 3 ]/ 64 
go (2,2) = [0DCT 0j0 + 0DCT lj0 + 0DCT 2jO + 0DCT 3 , 0 + 2DCT 0 ,i- 1DCT U - 2DCT2.1 
+ 3DCT 0 , 2 - 2DCT l)2 - 2DCT 0>3 ]/ 64 
35 go (3,2) = [ODCT 0 ,o+ 0DCT lj0 + 0DCT 2 , 0 + 0DCT 3 , 0 + 2DCT 0j i- 3DCT 1>1 + 2DCT2,! 
+ 3DCT 0>2 - 4DCT,, 2 - 2DCT 0 , 3 ]/ 64 
go (0,4) = [0DCT 0j0 + 0DCTi, 0 + 0DCT 2jO + ODCT 3)0 + 2DCT 0 ,]+ 3DCT U + 2DCT 2!l 
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- 3DCT 0 ,2- 4DCT U2 - 2DCT 0 , 3 ]/ 64 

go (1,4) - [ODCT 0 ,o+ 0DCTL0+ ODCTV0+ ODCT 3>0 + 2DCT 0 ,i+ 1DCT U - 2DCT\i 

- 3DCT 0i2 - 2DCT lj2 - 2DCT 0 , 3 ]/ 64 

go (2,4) = [ODCT 0 ,o+ ODCT UO + 0DCT 2 , 0 + 0DCT 3s0 + 2DCT 0 ,i- 1DCT U - 2DCT2,i 
5 - 3DCT 0> 2+ 2DCT 1)2 - 2DCT 0 , 3 ]/ 64 

go (3,4) = [ODCT 0 ,o+ ODCT UO + ODCT 2)0 + 0DCT 3 , 0 + 2DCT 0j i- 3DCT U + 2DCT2,i 

- 3DCT 0 ,2+ 4DCT 1)2 - 2DCT 0 , 3 ]/ 64 

go (0,6) = [0DCT 0j0 + 0DCTi, o + 0DCT 2j0 + 0DCT 3 , 0 + 1DCT 0j i+ IDCT lj+ IDCT^ 

- 3DCT 0i2 - 49DCT 1>2 + 6DCT 0>3 ]/ 64 

10 go (1,6) = [ODCT 0 ,o+ 0DCT U0 + ODCTV 0DCT 3jO +lDCT 0f i+0DCT u - lDCT^ 

- 3DCT 0>2 - 2DCT 1>2 + 6DCT 0>3 ]/ 64 

go (2,6) - [ODCT 0 ,o+ 0DCT U0 + 0DCT 2i0 + 0DCT 3 , 0 + 1DCT 0 ,i+ 0DCT u - lDCT^i 

- 3DCTo, 2 + 2DCT U + 6DCT 03 ]/ 64 

go (3,6) = [0DCT O)O + 0DCT 1)0 + 0DCT 2)0 + 0DCT 3i0 + 1DCT 0j1 - 1DCT U + lDCT^ 
15 - 3DCT 0 ,2+4DCT 1 , 2 + 6DCT 0)3 ]/ 64 

INTERLACED- SCAN. BASE-LAYER SET OF DECIMATED PIXEL VALUES 

gi (0,0) = [8DCT 0 ,o+ 7DCT lj0 + llDCT 0 ,i+ 10DCT U + 10DCT 0j2 + 0DCT^ o + 9DCT 0>3 

+ 8DCT 0 ,4+ 6DCT 0 , 5 + 4DCT 0j6 ]/ 64 
20 gi (1,0) = [8DCT 0 ,o- 7DCT lj0 + 1 1DCT 0 ,i- 10DCT U + 10DCT a2 + ODCT^* 9DCT a3 

+ 8DCT 0 ,4+ 6DCT 0 ,5+ 4DCT 0 , 6 ]/ 64 
g! (0,1) - [8DCT 0 ,o+ 7DCTi, 0 + 9DCT 0il + 9DCT U + 4DCT 0 , 2 + ODCTV 2DCT 0(3 

- 8DCT 0 ,4- 11DCT 0 ,5- 10DCT Os6 ]/64 

g! (1,1) = [8DCT 0)0 - 7DCT Ii0 + 9DCT 0)1 - 9DCT U + 4DCT 0 , 2 + 0DCT 2jO - 2DCT 0)3 
2 5 - 8DCT 0j4 - 1 1DCT 0i5 - 10DCT 0j6 ]/ 64 

g! (0,2) = [«DCT 0 ,o+7DCTi,o+ 6DCT 0 ,i+ 6DCT U - 4DCT 0)2 + 0DCT 2>0 - 11DCT 0 ,3 

- 8DCT 0)4 + 2DCT 0>5 + 10DCT 0j6 ]/ 64 

gi (1,2) = [8DCTo,o-7DCT lj0 + 6DCT 0 ,i- 6DCT U - 4DCT 0 , 2 + 0DCT 2 , 0 - HDCT 0i3 

- 8DCT 0i4 + 2DCT 0j5 + 10DCT o , 6 ]/ 64 

30 gi (0,3) = [8DCT 0f0 + 7DCT 1)0 + 2DCT 0 ,i+ 2DCT U - 10DCT 0?2 + ODCT^o- 6DCT a3 
+ 8DCT 0 ,4+ 9DCT 0 ,5- 4DCT 0)6 ]/ 64 
gi (1,3) = [8DCT 0>0 - 7DCT lj0 + 2DCT 0>1 - 2DCT U - 10DCT 0 , 2 + ODCIV 6DCT 0(3 

+ 8DCT 0 ,4+ 9DCT 0j5 - 4DCT a6 ]/ 64 
gi (0,4) = [8DCT 0 ,o+ 7DCT lj0 - 2DCT 0>1 - 2DCT U - 10DCT Of2 + 0DCT 2t0 + 6DCT 03 
35 + 8DCT 0 ,4- 9DCT 0i5 - 4DCT 0 , 6 ]/ 64 

g, (1,4) = [8DCT 0)0 - 7DCT 1>0 - 2DCT 0J + 2DCT U - 10DCT 0 , 2 + ODCT 2 ,o+ 6DCT 0 , 3 
+ 8DCT 0 ,4- 9DCT a5 - 4DCT 0;6 ]/ 64 
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gi (0,5) = [8DCT 0 ,o+ 7DCTi.o- 6DCT 0>1 - 6DCT U - 4DCT 0 , 2 + ODCT^ 1 1DCT 0 , 3 

- 8DCT 0 , 4 - 2DCTo, 5 + 10DCT 0j6 ]/ 64 

g! (1,5) - [8DCT 0j0 - 7DCT 1)0 - 6DCT 0 ,i+ 6DCT U - 4DCT 0)2 + ODCT i0 + 11DCT 0 , 3 

- 8DCT 0 ,4- 2DCT 0 ,5+ 10DCT 0>6 ]/ 64 

5 gi (0,6) = [8DCT 0 ,o+ 7DCT lj0 - 9DCT 0 ,i- 9DCT U + 4DCT 0 , 2 + 0DCT 2 , 0 + 2DCT 03 

- 8DCT 0 ,4+ 11DCT 0 ,5- 10DCT 0 , 6 ]/64 

g! (1,6) = [8DCT 0)0 - 7DCT 1>0 - 9DCT 0 ,i+ 9DCT U + 4DCT 0)2 + 0DCT 2 , o + 2DCT 0>3 

- 8DCT 0 ,4+ llDCTo.5- 10DCT 0)6 ]/64 

gi (0,7) = [8DCT 0 ,o+ 7DCT K0 - HDCT 0 ,i- 10DCT U +10DCT 0>2 + ODCT 2i0 ~ 9DCT 03 
10 + 8DCT 0 ,4- 6DCT 05 + 4DCT 0 , 6 ]/ 64 

gi (1,7) = [8DCT 0 ,o- 7DCTV 11DCT 0 ,i+ 10DCT U + 10DCT 0 , 2 + ODCTV 9DCT 0 , 3 
+ 8DCT 0 , 4 - 6DCTo, 5 + 4DCT 0 , 6 ]/ 64 

INTERLACED-SCAN. ENHANCEMENT-LAYER SET OF DECIMATED PIXEL VALUES 
15 go (0,0) = [0DCT o ,o+ 3DCT U0 + ODCT 0>1 + 4DCT U + 0DCT 0i2 + 7DCT2.0+ 0DCT 0i3 
+ 0DCT 0 ,4+ 0DCT 0 ,5+ ODCT 0 , 6 ]/ 64 
go (2,0) = [0DCT O)O + 3DCT,, 0 + 0DCT 0 ,i+ 4DCT U + 0DCT o>2 - 7DCT2, 0 + 0DCT 0 , 3 

+ 0DCT a4 + ODCT 0 ,5+ 0DCT 0i6 ]/ 64 
go (0,1) = [0DCT 0i0 + 3DCT li0 + 0DCT 0> i+ 4DCT U + 0DCT 0 , 2 + 7DCIV- 0DCT 03 
20 + 0DCT 0l 4+ 0DCT 0j5 + 0DCT 0)6 ]/ 64 

go (2,1) = [0DCT o ,o+ 3DCT lj0 + 0DCT 0 ,i+ 4DCT U + 0DCT o ^- 7DCT2, 0 + 0DCT 0 , 3 

+ 0DCT 0 ,4+ 0DCT 0i5 + 0DCT Oj6 ]/ 64 
go (0,2) = [ODCT 0 ,o+ 3DCT 1>0 + 0DCT 0J + 2DCT U + 0DCT Of2 + 7DCT2, 0 + 0DCT 0 , 3 
+ ODCTo,4+ 0DCT 0 , 5 + 0DCT 0 , 6 ]/ 64 
25 go (2,2) = [ODCT 0 ,o+ 3DCT 1>0 + 0DCT 0 ,i+ 2DCT U + 0DCT 0j2 - 7DCT2.0+ 0DCT 0 , 3 
+ 0DCT a4 + 0DCT 0 ,5+ 0DCT 0i6 ]/ 64 
go (0,3) = [0DCT O)O + 3DCT lj0 + 0DCT 0)1 + 1DCT U + 0DCT Ol2 + 7DCIV- 0DCT 0)3 

+ 0DCT 0 ,4+ 0DCT 0 ,5+ 0DCT 0 , 6 ]/ 64 
go (2,3) = [0DCT OjO + 3DCT li0 + 0DCT o ,i+ 1DCT U + 0DCT 0 , 2 - 7DCT2.0+ 0DCT a3 
30 + 0DCT 0 , 4 + 0DCTo, 5 + 0DCT 0 , 6 ]/ 64 

go (0,4) - [0DCT 0)0 + 3DCT U0 + 0DCT 0 .r 1DCT U + 0DCT a2 + 7DCT2, 0 + 0DCT 0 , 3 

+ 0DCT 0)4 + 0DCT o ,5+ 0DCT 0 , 6 ]/ 64 
go (2,4) - [ODCT 0 ,o+ 3DCT U0 + 0DCT 0 ,i- 1DCT U + 0DCT o , 2 - 7DCT 2 , 0 + 0DCT 0 , 3 
+ 0DCT o ,4+ 0DCT o>5 + 0DCT a6 ]/ 64 
35 go (0,5) = [ODCT 0 ,o+ 3DCT li0 + ODCT 0 ,i- 2DCT U1 + 0DCT Oj2 + 7DCT2, 0 + 0DCT 0>3 
+ 0DCT 0i4 + 0DCT 0 ,5+ 0DCT 0 , 6 ]/ 64 
go (2,5) = [ODCT 0l o+ 3DCT 1>0 + ODCT 0> i- 2DCT U + 0DCT 0 , 2 - 7DCT 2t0 + ODCT 0(3 
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+ ODCT 0>4 + 0DCT O5 + 0DCT 0 , 6 ]/ 64 
go (0,6) = [ODCT 0t0 + 3DCT 1>0 + ODCT 0 .r 4DCT U1 + 0DCT 0i2 + 7DCT2, 0 + 0DCT 0j3 

+ 0DCT 0i 4+ 0DCT o ,5+ 0DCT 0 , 6 ]/ 64 
go (2,6) = [ODCT 0 ,o+ 3DCT 1>0 + ODCT 0t i- 4DCT U + 0DCT o>2 - 7DCT 2 , 0 + 0DCT 0i3 
5 + 0DCT 0 ,4+ 0DCT o ,5+ 0DCT 0 , 6 ]/ 64 

go (0,7) = [ODCT 0)0 + 3DCT no + ODCT 0 ,i- 4DCT U + ODCT 0 , 2 + 7DCT2, 0 + 0DCT o , 3 

+ 0DCT 0 ,4+ ODCTo, 5 + 0DCT 0 , 6 ]/ 64 
go (2,7) = [0DCT ao + 3DCT 1>0 + 0DCT o ,i- 4DCT U + 0DCT Oi2 - 7DCT2, 0 + 0DCT 0 , 3 

+ 0DCT 0f4 + ODCT a5 + 0DCT O)6 ]/ 64 

10 

Each of the above "Progressive-Scan Set of Decimated Pixel Values" and above 
"Interlaced-Scan Set of Decimated Pixel Values" was derived in the following manner: 
1 . If DCT M , V denotes the DCT coefficient with horizontal frequency index u and 
vertical frequency index v, then the IDCT equation which would be used to 

15 decode a block denoted f(x,y) at full resolution (where x = 0, ,N-1; y = 

0, ,N-l)is given by 

* 2 N-1N~1 (2x + 1)uti (2y + l)v7i /1X 

( ?y) "N u £ov£o (U) (V) DCT ^ vC0S 2N C ° S 2N 0) 



20 2. Using only the 10 DCT coefficients shown in FIGURE lb, gives the 

approximation equation 2 for progressive-scan sequences 



I DCT 0,0 + ^DCT li0 co.^)JL + 

1 DCT 2 (2xH-l)2, + 1 (2x + l) 37I + 

1 nrr _ (2y + l)7t , (2x + 1)tc (2y + l>7i 

V^ DCT 0,l TOS ^2^- +DCT ia COsl -2N C ° S 2N + (2) 
1 _ (2y + l)27c^ r .„„ (2x + l)2:t (2y + l)jt 

DCTl 2 cos <^cos + * DCT 0 3 cos 

1 > Z 2N 2N Jy °' j 2N 



f(x,y)« 



N 



25 3. Using only the 10 DCT coefficients shown in FIGURE lc, gives the 

approximation equation 3 for interlaced-scan sequences 
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l nrT , 1 nrT „(2x + l)7i 1 „™ (2x + 1)2tc 

2 DCT 0,0 + ^ DCT l ; Q^s-^ + - /r DCT 2?0 cos ^ ^ + 

V2 U > A 2N U 2N 2N 

1 nrT — (2y + 1)271 ^ 1 ^™ (2y-H)37i 
— =DCl n 9 cos J ^/ — + — =^DCT n ocos — — + 

V2 2N V2 °' 3 2N 

1 (2y + 1)4tc, 1 ^™ (2y + l)57t 

1 nr-r (2y + l)6?i 



(3) 



4. Let the right-hand side of each of equations 2 and 3 be denoted f (x,y). In the 
case of a progressive scan (i.e., the progressive_sequence flag is 1), the base- 
5 layer value gi'(x,y) is computed in accordance with the following equation 4 

and the enhancement-layer value go'( x >y) is computed in accordance with the 
following equation 5: 



M x > y) = i[ f '( 2x > 2 y) + f '< 2x + ! > 2 y) + f \2x,2 y +i)+f\2 X +i,2y+i)] 



(4) 



10 forx = 0,....,3;y = 0,....,3. 

g 0 , (x 5 y) = I[fV2x,y)+f , (2x + l 5 y)-f r (2x,y + l)-f , (2x + l,y + l) 

fbrx = 0,....,3;y = 0, 2, 4, 6. 



(5) 



More specifically, gi'(x,y) in equation 4 defines the average value of the values 
15 of a set of 4 contiguous pixels (or prediction errors) arranged in a 2x2 block 

portion of the full-resolution 8x8 block. The value go'( x >y) in equation 5 defines 
the difference between the average value of the values of a first set of 2 
contiguous horizontal pixels (or prediction errors) of one vertical line and the 
average value of the values of a second set of 2 contiguous horizontal pixels (or 
20 prediction errors) of the following vertical line arranged in a 2x2 block portion 

of the full-resolution 8x8 block. The 16 equations gi(0,0) to gi(3,3) of the 
above "Progressive-Scan, Base-layer Set of Decimated Pixel Values" were 
derived by substituting equation 2 into equation 4, substituting numeric values 
for x and y in gi'(x,y), substituting N = 8, and approximating the weighting 
25 factors for the DCT coefficients with rational values. The 16 equations go(0,0) 

to go(3,6) of the above "Progressive-Scan, Enhancement-layer Set of Decimated 
Pixel Values" were derived in a similar manner by substituting equation 2 into 
equation 5, substituting numeric values for x and y in go(x,y), substituting N = 
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8, and approximating the weighting factors for the DCT coefficients with 
rational values. Although the effective pixel decimation of the enhancement- 
layer is only 2 (rather than being the effective pixel decimation of 4 of the base- 
layer), the equalities g 0 (x,y+l) = -go(x,y) hold for y = 0,2,4,6, so that 
5 enhancement-layer values with odd vertical indexes need not be computed. 

Thus, only 16 independent go(x,y) enhancement-layer values need be computed 
for each 8x8 luma block in a progressive-scan I or P picture. Further, because 
these 16 go(x,y) enhancement-layer values are residual values, they tend to have 
a small dynamic range. 

10 In the case of an interlaced scan (i.e., the progressive_sequence flag is 

0), the base-layer value gi'(x,y) is computed in accordance with the following 
equation 6 and the enhancement-layer value go'(x,y) is computed in accordance 
with the following equation 7: 



15 g 1 , (x,y) = i[f'(4x,y) + f , (4x + l,y) + f'(4x + 2 ? y) + f , (4x + 3,y) 
forx = 0,l; y = 0,....,7. 

for x = 0,2; y = 0,....,7. 



(6) 



(7) 



20 In the interlaced-scan case of an 8x8 block, gi'(x,y) in equation 6 defines the 

average value of the values of a set of 4 contiguous pixels (or prediction errors) 
arranged in a 4x1 block portion of the 8x8 block. The value go'(x,y) in equation 
7 defines the difference between the average value of the values of a first set of 
2 contiguous horizontal pixels (or prediction errors) of a vertical line and the 

25 average value of the values of a second set of the next 2 contiguous horizontal 

pixels (or prediction errors) of the same vertical line arranged in a 4x1 block 
portion of an 8x8 block. The 16 equations gi(0,0) to gi(l,7) of the above 
"Interlaced-Scan, Base-layer Set of Decimated Pixel Values" were derived by 
substituting equation 3 into equation 6, substituting numeric values for x and y 

30 in gj'(x,y), substituting N = 8, and approximating the weighting factors for the 

DCT coefficients with rational values. The 16 equations go(0,0) to go(2,7) of the 
above "Interlaced-Scan, Enhancement-layer Set of Decimated Pixel Values" 
were derived in a similar manner by substituting equation 3 into equation 7, 
substituting numeric values for x and y in go'(x,y), substituting N = 8, and 

35 approximating the weighting factors for the DCT coefficients with rational 

values. Although the effective pixel decimation of the enhancement-layer is 
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only 2 (rather than the effective pixel decimation of 4 of the base-layer), the 
equalities g 0 (x+l,y) = -go(x,y) hold for x = 0 and x = 2 so that enhancement- 
layer values with odd horizontal indexes need not be computed. Thus, only 16 
independent go(x,y) enhancement-layer values need be computed for each 8x8 
5 luma block in an interlaced-scan I or P picture. Further, because these 16 go(x,y) 

enhancement-layer values are residual values, they tend to have a small 
dynamic range. 

Returning to FIGURE 2, unit 204 conveys an output comprising successive 8x8 
10 blocks of I, P and B luma and chroma gi(x,y) base-layer decimated pixel values as a 
first input to base-layer adder 205B in a predetermined order. (For non-coded blocks all 
such values are zero). This predetermined order includes the decimated pixel values of 
each 2x2 array of 8x8 pixel luma blocks and each of two chroma blocks which form a 
decimated macroblock for use by enhanced MCU processing means 208. Further, unit 
15 208 applies a corresponding block pi(x,y) of base-layer decimated pixel values as a 
second input to base-layer adder 205B in this same predetermined order (For intra- 
coded macroblocks all such values are zero). The block si(x,y) of base-layer decimated 
pixel values derived as a sum output from base-layer adder 205B are then stored in 
memory 206. 

20 Unit 204 conveys an output comprising the I and P luma go(x,y) enhancement- 

layer decimated pixel values as a first input to enhancement-layer adder 205E in the 
previously mentioned decimated-pixel macroblock predetermined order. (For non- 
coded blocks all such values are zero). Further, for the case of P luma pixels, unit 208 
applies a corresponding macroblock of 64 po(x,y) enhancement-layer decimated pixel 

25 values as a second input to adder 205E in this same predetermined order. (For intra- 
coded macroblocks all such values are zero). The macroblock of 64 So(x,y) 
enhancement-layer decimated pixel values derived as a sum output from adder 205E 
are applied as an input to enhancement-layer encoder 207 and then the encoded output 
bit-words from encoder 207 are stored in memory 206 during decoding of I and P 

30 pictures. 

A macroblock at the higher resolution of the enhancement-layer would 
normally comprise 128 decimated luma pixel values. However, because of the above- 
described symmetry equalities for both progressive-scan sequences and interlaced-scan 
sequences, the number of independent decimated enhancement-layer pixel values in the 
35 block s 0 (x,y) is reduced from 128 to 64. Therefore, the predetermined order is such that 
only half of the enhancement-layer decimated pixel values need be considered by 
enhancement-layer encoder 207. These enhancement-layer values are encoded in pairs 
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using a simple vector quantizer, with each pair of values being represented by an 8-bit 
codeword. Since there are 64 enhancement-layer values to be encoded in a macroblock, 
the number of bits of storage for the enhancement layer is 32x8 = 256 bits per 
macroblock. In the preferred embodiment the 32 codewords are combined into two 
5 128-bit output words from encoder 207 for storage in memory 206. 

For progressive sequences each pair of horizontally adjacent values in the block 
s 0 (x,y) is encoded as a two-dimensional vector, whereas for interlaced sequences each 
pair of vertically adjacent (within the same field) values in So(x,y) is encoded as a two- 
dimensional vector. Let v 0 and vi be a pair of values to be encoded together. The 

10 computational procedure employed by encoder 207 to encode the pair v 0 ,vi is described 
in detail in Appendix A. After this procedure has been completed for each pair of 
values in s 0 (x,y), the codewords are packed into two 128-bit words, both of which 128- 
bit words form the output from encoder 207 that are stored in memory 206. 
Returning again to FIGURE 2, memory 206 provides (1) a base-layer output di(x,y) to 

15 unit 208 (di(x,y) is similar in content to the base-layer input si(x,y) provided to 
memory 206) and (2) an enhancement-layer output to unit 208 (similar in content to the 
enhancement-layer input to memory 206). 

In order for enhanced MCU processing means 208 to form a block of 
predictions, a block of pixel values is fetched from memory 206. The base-layer of 

20 pixel values which are read from the stored reference picture are denoted di(x,y). The 
enhancement-layer residual values, which are needed only if the block of predictions 
being formed is for the luma component in a P picture, are denoted d 0 (x,y). Since the 
enhancement-layer samples are stored in memory 206 in encoded form, the 
enhancement-layer data output from memory 206 input to unit 208 is decoded by 

25 enhancement-layer decoder 300 (Figure 3) to obtain the d 0 (x,y) values. Unit 208 
separately forms individual luma or chroma outputs for field prediction operations 
corresponding to the top and bottom field prediction blocks. In a bi-directionally 
predicted macroblock these operations are performed separately for the forward and 
backward predictions and the results are combined as described in the ISO 13818-2 

30 standard. In the following detailed description of the computationalrprocessing 
operations performed by unit 208, the symbol / represents integer division with 
truncation of the result toward minus infinity, the symbol // represents integer division 
with truncation of the result toward zero, and the symbol % represents the modulus 
operator, which is defined such that if x is a negative number and M is a positive 

35 number, then x%M=M-((x//M)*M-x). 

Before a block of samples can be read from memory 206, the location and size 
of the block is determined. The location of a block of pixel values in the reference 
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picture is specified by the horizontal and vertical coordinates of the start (i.e., the 
upper-left corner) of the block in the reference picture. For the base-layer, these 
coordinates are indexes into a picture which is 1/4 horizontal, full vertical resolution for 
interlaced sequences and 1/2 horizontal, 1/2 vertical resolution for progressive 
5 sequences. For the enhancement-layer, the coordinates are indexes into a picture which 
is 1/2 horizontal, full vertical resolution for both interlaced and progressive sequences. 

To locate the blocks di(x,y) and d 0 (x,y) in the reference picture, the motion 
vector for the macroblock being decoded is needed. The decoding of motion vector 
data in the bitstream, the updating of motion vector predictors, and the selection of 

10 motion vectors in non-intra macroblocks which contain no coded motion vectors (e.g., 
skipped macroblocks) are all performed by unit 208 as described in the ISO 13818-2 
standard. Let x b and y b be the full-resolution horizontal and vertical positions of the 
macroblock being decoded and let mv=(dx,dy) be the decoded motion vector, so that if 
the sequence were being decoded at full resolution, a block of pixel values at location 

15 (xb+(dx/2), y b +(dy/2)) in the full-resolution reference luma picture would be read from 
memory and used to form luma predictions. Similarly, a block of chroma values at 
location (xb/2+(dx//2), yb/2+(dy//2)) in the reference chroma picture would be needed 
to form predictions for each of the 2 chroma components in a full-resolution mode. 

The location in the reference picture of a block needed for motion compensation 

20 in unit 208 is determined using x b , y b , dx and dy. Table 3, shows the locations of blocks 
for various prediction modes. The sizes of the blocks needed for motion compensation 
in unit 208 are specified in Table 4. Base-layer entries in Table 4 give the size of the 
block di(x,y), and enhancement-layer entries in Table 4 give the size of the block 
do(x,y). 
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Prediction Mode 


Horizontal 
Coordinate 


Vertical 
Coordinate 


Progressive sequence, luma, 
base-layer 


((xb+(dx/2))/8)*4 


(y b +(dy/2))/2 


Progressive sequence, luma, 
enhancement-layer 


((xb+(dx/2))/8)*4 


((y b +(dy/2))/2)*2 


Progressive sequence, chroma 


Xb/4+((dx//2)/4) 


y b /4+((dy//2)/4) 


Interlaced sequence, luma, 
base-layer 


((x b +(dx/2))/8)*2 


Vb+(dy/2) 


Interlaced sequence, luma, 
enhancement-layer 


((x b +(dx/2))/8)*4 


Vb+(dy/2) 


Interlaced sequence, chroma 


Xb/8+((dx//2)/8) 


yb /2+((dy//2)/2) 



Table 3 -Locations of Blocks Needed for Motion Compensation 
in Enhanced MCU Processing Means 208 



Prediction Mode 


Horizontal 
Size 


Vertical 
Size 


Progressive sequence, luma,base-layer 


12 


9 


Progressive sequence, luma,enhancement-layer 


12 


18 


Progressive sequence, chroma 


5 


5 


Interlaced sequence, luma, 16x16 prediction,base-layer 


6 


17 


Interlaced sequence, luma, 16x8 prediction, base-layer 


6 


9 


Interlaced sequence, luma, 16x16 prediction, 
enhancement-layer 


12 


17 


Interlaced sequence, luma, 16x8 prediction, 
enhancement-layer 


12 


9 


Interlaced sequence, chroma, 8x8 prediction 


3 


9 


Interlaced sequence, chroma, 8x4 prediction 


3 


5 



Table 4-Sizes of Blocks Needed for Motion Compensation 
in Enhanced MCU Processing Means 208 

10 

Figure 3 shows the processing performed on the luma samples read from 
memory 206 by unit 208. As shown in Figure 3, the luma-processing portion of unit 
208 comprises enhancement-layer decoder means 300, enhancement-layer pixel 
reconstruction means 302, DCT-based upsample means 304, full-resolution block 
15 select means 306, DCT-based downsample means 308 and two-layer output formation 
means 310. These elements of enhanced MCU processing means 208 use the reduced- 
resolution blocks from I and P frames stored in memory 206 to form predictions for a 
decoded macroblock. 
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The above-described structure of Figure 3 performs computational processing 
of luma pixel values input to unit 208 from memory 206. This computational process is 
described in detail in appendices B to G. Briefly, however, decoder 300 unpacks the 
input 128-bit words into 16 constituent 8-bit codewords. Decoder 300, employing the 
5 computational processing described in appendix B, derives d 0 (x,y) as an output. 
Enhancement-layer pixel reconstruction means 302, employing the computational 
processing described in appendix C, derives r 0 (x,y) as an output in response to both the 
di(x,y) input to unit 208 and the d 0 (x,y) output from decoder 300. DCT-based upsample 
means 304, employing the computational processing described in appendix D, 

10 horizontally upsamples the r 0 (x,y) input to derive r(x,y) at full resolution. Full- 
resolution block select means 306, employing the computational processing described 
in appendix E, uses r(x,y) to derive a full-resolution block of predictions p(x,y) as an 
output. DCT-based downsample means 308, employing the computational processing 
described in appendix F, horizontally downsamples the p(x,y) input to derive q(x,y) at 

15 half horizontal resolution. The block q(x,y) is applied as an input to two-layer output 
formation means 310, which employs the computational processing described in 
appendix G, to derive the outputs pi(x,y) and po(x,y) provided by unit 208 to adders 
205B and 205E shown in FIGURE 2. Although the computational processing required 
for chroma predictions are not shown in FIGURE 3, they are described in detail in 

20 appendix H. 

Returning again to FIGURE 2, a video signal comprising the base-layer pixels 
defining each of successive picture fields or frames is output from memory 206 and 
input to sample-rate converter 210 which derives a display video signal output. The 
display video signal output from unit 210 represents a PIP display image. By way of 

25 example, assume that the size of a PIP display is intended to occupy 1/3 of the 
horizontal dimension and 1/3 of the vertical dimension of the entire HD display size. If 
the original resolution of the HD bit-stream is 1920x1080 interlaced, then the PIP 
decoded frames (in which the number of pixels in the horizontal direction has been 
decimated by a factor of 3/4) are 480x1080 interlaced. Assuming a 1920x1080 

30 interlaced HD display, the displayed PIP frames should be 640x360 interlaced. 
Therefore, in this example, the decoded frames stored in memory must be scaled by 
sample-rate converter 210 by a factor of 4/3 in the horizontal direction and 1/3 in the 
vertical direction. 

In a realized embodiment of the present invention, the extra capacity required in 
35 1 memory for storage of the 1/2 resolution enhancement-layer in encoded form adds 
only 1.98 Mbits to the 17.8 Mbits required for storage of the 1/4 resolution base-layer. 
Thus, the inclusion of an encoded 1/2 resolution enhancement-layer increases the 
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needed storage capacity of the base and enhancement-layer decimated-pixel 
by a relatively small amount (i.e., only a little more than 1 1%) to 19.78 Mbits. 
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APPENDIX A 
Enhancement-layer Encoder 207 



In the following procedure, used to encode each pair of enhancement-layer 
5 values v 0 ,v u the symbol "DIV" represents integer division with rounding of the result 
to the nearest integer. 

First, the values v 0 and vi are clipped to the range [-45,45]. Then the 8-bit 
codeword C is computed as shown in the following pseudocode: 



10 if (v 0 > -4 AND v 0 < 4 AND v t > -4 AND vi < 4) 

C = 7*(vi +3) + v 0 + 211 

else 

if(vi<v 0 -25) 

v 0 = (v 0 + vi + 25)/2 
15 vi = v 0 - 25 

else if (vi > v 0 + 25) 

v 0 = (v 0 + vi - 25)/2 
vi = v 0 + 25 
v 0 = 5*(v 0 DIV 5) 
20 vi = 5*(vi DIV 5) 

C = 104 - 2*vi - vo/5 



The codeword C is the value stored in memory to represent the pair vo,vi. 
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APPENDIX B 



Enhancement-layer Decoder 300 



After a 128-bit enhancement-layer word read out from base and enhancement- 
layer memory 206 has been unpacked into 16 separate 8-bit codewords, each codeword 
can be decoded as described below. Let C be a codeword to be decoded and let b 0 and 
bi be the values to be obtained by decoding C. Then the following pseudocode shows 
how b 0 and bi should be computed: 

if(C<204) 

b 0 = 70-5*(C/ll)-5*(C%ll) 
bi = 45 - 5*(C/1 1) 



Each decoded codeword is used to fill in a portion of do(x,y). For progressive 
sequences, lines of do(x,y) which come from odd-indexed lines in the reference picture 
are obtained by negating the appropriate decoded codeword values. Similarly, for 
interlaced sequences, columns of d 0 (x,y) which come from odd-indexed columns in the 
reference picture are obtained by negating the appropriate decoded codeword values. 



else 



bo 
b, 



(C 
(C 



208)°/o7 - 3 
208)/7 - 3 
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APPENDIX C 
Enhancement-layer Pixel Reconstruction Means 302 



Using the block d 0 (x,y) output from enhancement-layer decoder 300 and block 
5 di(x,y) read out from base and enhancement-layer memory 206, the pixel values at the 
enhancement-layer resolution (i.e., ^-horizontal, full vertical resolution) can be 
reconstructed for both P pictures and B pictures. For P pictures d 0 (x,y) is obtained as 
described in Appendix B; for B pictures do(x,y) is assumed to be 0 for all x and y. 

The equations used to combine the 2 layers for progressive sequences are given 

10 by 



r 0 (x,2y) = di(x,y) + do(x,2y) 
r 0 (x,2y + 1) = di(x,y)+ d 0 (x,2y + 1) 

15 fbrx = 0,...,ll,y = 0,...,8. 

The equations used to combine the 2 layers for interlaced sequences are given 

by 



r 0 (2x, y) = di(x, y) + do(2x,y) 
20 r 0 (2x + l,y) = di(x,y) + do(2x + l,y) 

where x = 0,...,5; y = 0,...,8 for 16x8 prediction and x = 0,...,5; y = 0,...,16 for 16x16 
prediction. 

The block r 0 (x,y), obtained using the above equations, contains the 
25 enhancement-layer pixels at one-half horizontal resolution, so that the enhancement- 
layer pixels still need to be upsampled by a factor of 2 horizontally to reach full 
resolution. 
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APPENDIX D 
DCT-Based Up sample Means 304 

DCT-based upsample means 304 linearly transforms the 12 input pixel values 
5 in each row of r 0 (x,y), to 24 output pixel values representing each row of r(x,y) at full 
resolution. As shown in FIGURE 4, the 12 pixel values in a row are computationally 
processed in 3 groups of 4, where the 12 input pixel values are divided into 3 groups 
Wi, W 2 and W 3 . For each of the 3 groups, the DCT is a 4-point DCT, followed by a 
zero pad that adds 4 zeros to the end of that 4-point DCT output to result in an 8-point 
10 output from that zero pad. The IDCT for each of the 3 groups is an 8-point IDCT, 
which results in the 24 output pixel values representing each row of r(x,y) at full 
resolution comprising 3 groups Z h Z 2 and Z 3 , wherein the output pixel values for each 
of the 3 groups consists of 8 pixel values. 

The linear transformation shown in Figure 4 can be computationally 
15 implemented as 3 matrix-vector multiplications, given by the equations Zj = (l/64)AWj 
for j = 1,2, and 3, where 



A = 
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The result of performing this linear transformation on each row of input pixel 
values in r 0 (x,y) is the block of r(x,y) output pixel values at full resolution. 
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APPENDIX E 
Full-Resolution Block Select Means 306 

A final 16x16 or 16x8 full-resolution block of predictions, denoted p(x,y), must 
5 be obtained from r(x,y), using the motion vector (dx,dy) to select a subset of the pixels 
in r(x,y), with half-pixel interpolation if necessary. More specifically, if the upper-left 
pixel in r(x,y) has coordinates (0,0) and the coordinates of r(x,y) are specified with 
half-pixel precision, then the upper left corner of the final prediction block is 
r(dx%16,dy%4) for progressive sequences and r(dx%16,dy%2) for interlaced 
10 sequences. Since the coordinates of r(x,y) are specified with half-pixel precision, odd 
values of dx%M or dy%M (for M = 2,4,16) imply that half-pixel interpolation, as 
described in the ISO 13818-2 standard, must be performed. 
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APPENDIX F 
DCT-Based Downsample Means 308 



DCT-based downsample means 308 linearly transforms each row of p(x,y) from 
16 pixels to 8 pixels. As shown in FIGURE 5, the 16-pixel row is first divided into two 
groups of 8 pixels, Ui and U 2 . For each of the 2 groups, the DCT is an 8-point DCT, 
followed by a truncation which discards the last 4 points in the DCT. The IDCT for 
each of the 2 groups is a 4-point IDCT, which results in the 8 output pixel values 
representing each row of q(x,y) at one-half resolution comprising 2 groups Vi and V 2 , 
wherein the output pixel values for each of the 2 groups consists of 4 pixel values. The 
computationally-processed output of DCT-based downsample means 308 shown in 
FIGURE 5 is a set of 8 pixels, formed from the concatenation of Vi and V 2 . 

The linear transformation shown in Figure 5 can be computationally 
implemented as 2 matrix-vector multiplications, given by the equations Z s = BUj for j = 
1 and 2, where 



f38 
1-9 



12 



6-3-2 2 1 -1] 
30 28 9 -6 -4 4 I 
-6 9 28 30 12 -9 I 




1 2 -2 -3 6 23 38 J 



The result of performing this linear transformation on each row of input pixel 
values in p(x,y) at full resolution is the block of q(x,y) output pixel values at one-half 
horizontal resolution. 
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APPENDIX G 
Two-Layer Output Formation Means 310 



In the following procedure, the symbol "DIV" represents integer division with 
5 rounding of the result to the nearest integer. The base-layer output of the MCU, 
denoted pi(x,y), is obtained for progressive sequences using the equation 



Pi(x,y) - (q(x,2y) 4- q(x,2y+l)) DIV 128 



10 forx = 0,....,7;y = 0,....7. 

The base-layer output of the MCU, denoted pi(x,y), is obtained for interlaced 
sequences using the equation 



Pl (x,y) = (q(2x,y) + q(2x + l,y)) DIV 128 

15 

for x = 0,.,..,3; y = 0,....,7 for 16x8 prediction and x = 0,.,..,3; y = 0,....,15 for 16x16 
prediction. 

The enhancement-layer output of the MCU, denoted po(x,y) and computed only 
for P pictures, is obtained for progressive sequences using the equation 

20 

Po(x,y) = (q(x,y) - q(x,y + 1)) DIV 128 



forx = 0,....,7;y = 0,2,4,....,12,14. 

The enhancement-layer output of the MCU, denoted po(x,y) and computed only 
25 for P pictures, is obtained for interlaced sequences using the equation 



po(x,y) = (q(x,y) - q(x + l,y)) DIV 128 



for x = 0,2,4,6; y = 0,....,7 for 16x8 prediction and x = 0,2,4,6; y = 0,....,15 for 16x16 
30 prediction. 

As shown in FIGURE 2, the MCU base-layer output pi(x,y) is applied as an 
input to adder 205B and the MCU enhancement-layer output po(x,y) is applied as an 
input to adder 205E. 
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APPENDIX H 
Forming Block of Predictions for the Chroma Component 

In the following procedure, the symbol "DIV represents integer division with 
5 rounding of the result to the nearest integer. If the block of predictions being formed is 
for the chroma component, then the samples di(x,y) are upsampled to full resolution 
using a pixel repeat operation. Then the final full resolution 8x8 or 8x4 block of 
predictions is selected using the motion vector (dx,dy), with half-pixel interpolation, if 
necessary. Finally, the full-resolution predictions are downsampled to provide base- 
10 layer predictions to be stored in base and enhancement-layer memory 206. 

The pixel repeat operation, which transforms the base-layer block di(x,y) to the 
full-resolution block r(x,y), is performed for progressive sequences using the equation 



15 



r(x,y) = di(x/2,y/2) 

fcrx = 0,....,9;y = 0,....,9. 

The pixel repeat operation, which transforms the base-layer block di(x,y) to the 
full-resolution block r(x,y), is performed for interlaced sequences using the equation 

20 r(x,y) = di(x/4,y) 

for x = 0,...,!!; y = 0,...,4 for 8x4 prediction and x = 0,...,11; y = 0,...,8 for 8x8 
prediction. 

The final 8x8 or 8x4 full-resolution block p(x,y) is obtained from r(x,y). The 
25 upper-left corner of this final block is r((dx//2)%4,(dy//2)%4) for progressive 
sequences and r((dx//2)%8,(dy//2)%2) for interlaced sequences. Odd values of 
(dx//2)%M or (dy//2)%M (for M = 2,4,8) imply half-pixel interpolation is necessary. 

The base-layer prediction block pi(x,y) is obtained by downsampling p(x,y). 
For progressive sequences the equation used to perform this downsampling is 



30 



35 



pi(x,y) = (p(2x,2y) + p(2x + l,2y) + (p(2x,2y + 1) + p(2x + l,2y + 1)) DIV 4 

for x = 0,....,3; y = 0,....,3. For interlaced sequences the equation used to perform this 
downsampling is 

Pi(x,y) = (P(4x,y) + p(4x + l,y) + p(4x + 2,y) + p(4x + 3,y)) DIV 4 
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for x = 0,1; y = 0,...,3 for 8x4 prediction and x = 0,1; y = 0,....,7 for 8x8 prediction. 

As shown in FIGURE 2, the MCU base-layer output pi(x,y) is applied as 
input to adder 205B. 
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WHAT IS CLAIMED IS: 

1 . In apparatus for decoding compressed image data including frequency 
domain coefficients defining blocks of pixel values representing an image at a first 

5 resolution to provide an image at a reduced second resolution for display, said 
apparatus comprising: 

first means responsive to a selected sub-set of said frequency domain 
coefficients for deriving said image of said reduced second resolution for display and 
including, 

10 enhanced motion-compensation-unit (MCU) processing means; 

and 

second means for operating said enhanced MCU processing 
means with blocks of pixel values representing said image at an intermediate third 
resolution lower than said first resolution and higher than said reduced second 
15 resolution. 

2. The apparatus defined in Claim 1, wherein said reduced second 
resolution is substantially 1/4 of said first resolution; and 

said second means operates said enhanced MCU processing at an 
20 intermediate third resolution which is substantially 1/2 of said first resolution. 

3. The apparatus defined in Claim 1, wherein said image at said reduced 
second resolution for display is a progressive-scanned image. 

25 4. The apparatus defined in Claim 1, wherein said image at said reduced 

second resolution for display is an interlaced-scanned image. 

5. The apparatus defined in Claim 1, wherein: 

said enhanced MCU processing means is responsive to base-layer pixel 
30 macroblock input values representing said image at said reduced second resolution and 
to pixel values representing said image at said intermediate third resolution for deriving 
motion-compensated base-layer prediction macroblock output pixel values as a first 
output and motion-compensated enhancement-layer prediction macroblock output pixel 
residual values as a second output. 

35 
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6. The apparatus defined in Claim 5, wherein: 

said second means comprises third means responsive to said selected 
sub-set of said frequency domain coefficients and to both said motion-compensated 
base-layer macroblock output pixel values and said enhancement-layer macroblock 
output pixel residual values for deriving both said base-layer macroblock input pixel 
values and said encoded enhancement-layer macroblock input pixel residual values. 

7. The apparatus defined in Claim 1, wherein said second means 

comprises: 

a base and enhancement-layer decimated-pixel memory; 

unitary enhanced inverse discrete cosine transform (IDCT), filtering and 
pixel-decimation processing means responsive to a selected sub-set of frequency 
domain coefficients for deriving base-layer blocks of output pixel values representing 
said image at said reduced second resolution as a first output and output enhancement- 
layer blocks of output pixel residual values representing said image at said intermediate 
third resolution as a second output; 

fourth means, including a first adder for adding corresponding pixel 
values of said motion-compensated base-layer macroblock output pixel values from 
said enhanced MCU processing means and said base-layer blocks of output pixel 
values from said unitary IDCT, filtering and pixel-decimation processing means, for 
deriving values that are stored as base-layer data in said base and enhancement-layer 
decimated-pixel memory; 

fifth means, including a second adder and an enhancement-layer 
encoder, for adding corresponding pixel residual values of said motion-compensated 
enhancement-layer macroblock output pixel residual values from said enhanced MCU 
processing means to said enhancement-layer blocks of output pixel residual values 
from said unitary IDCT, filtering and pixel-decimation processing means to obtain a 
sum output from said second adder for encoding by said enhancement-layer encoder, 
for deriving second input values that are stored as encoded enhancement-layer data in 
said base and enhancement-layer decimated-pixel memory; and 

sixth means for providing from said base and enhancement-layer 
decimated-pixel memory said base-layer pixel macroblock input values to said 
enhanced MCU processing means and for deriving said encoded enhancement-layer 
pixel macroblock input residual values applied as a second input to said enhanced 
MCU processing means from said stored encoded enhancement-layer data. 
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8, The apparatus defined in Claim 1, wherein: 

said frequency domain coefficients define image information that 
includes luma blocks of pixel values representing intracoded (I) and predictive-coded 
(P) progressive-scanned images at said first resolution. 

5 

9. The apparatus defined in Claim 7, including: 

seventh means comprising a sample-rate converter for deriving an 
ongoing display video signal from base-layer blocks of output pixel values. 

10 10. The apparatus defined in Claim 1, wherein: 

said reduced second resolution is substantially 1/4 of said first 

resolution; and 

said intermediate third resolution is substantially 1/2 of said first 

resolution. 

15 

1 1. In a system for decoding compressed image data in the form of pixel 
blocks representing an image at a first resolution to provide an image of a reduced 
second resolution, a method comprising the steps of decompressing a pixel block of 
said first resolution by: 

20 selecting a sub-set of frequency domain coefficients in said pixel blocks 

of said compressed image data; 

processing elements of said sub-set of frequency domain coefficients to 
provide pixel data representing pixels comprising a spatially distributed sub-set of 
pixels in a pixel block of said image at a first resolution and excluding other pixels of 
25 that pixel block, said processing including 

using data at an intermediate third resolution, lower than said 
first resolution but higher than said reduced second resolution, to supplement data from 
said reduced second resolution in forming predictions for motion compensation; and 

formatting said pixel data representing pixels comprising said spatially 
30 distributed sub-set of pixels to provide said image of said reduced second resolution. 



12. A method according to claim 11 including the step of, 
selecting different spatially distributed sub-sets of pixels for 
interlace and progressive image output. 

35 
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13 . A method according to claim 11 wherein said formatting step 

comprises, 

upsampling said pixel data representing pixels comprising a 
spatially distributed sub-set of pixels to provide said image of said reduced 
5 second resolution. 

14. A method according to claim 1 1 wherein said processing step 
includes the step of, 

selecting said spatially distributed sub-set of pixels based on 
10 desired PIP picture characteristic. 

15. A method according to claim 14 wherein said PIP picture 
characteristic comprises at least one of (a) PIP picture size, (b) whether said PEP 
picture is interlace or progressive, and (c) PIP picture vertical and horizontal 

15 pixel resolution. 

16. A method according to claim 1 1 wherein said formatting step 
includes the step of, 

adaptively filtering pixel data representing pixels comprising a 
20 spatially distributed sub-set of pixels using a filter function selected based on at 
least one of, (a) motion vector type, (b) group of picture (GOP) structure, (c) a 
GOP boundary transition, (d) whether I, B or P frame, and (e) whether interlace 
or progressive frame reduced second resolution output required. 

25 17. A method according to claim 1 1 wherein said formatting step 

includes the step of, 

adaptively filtering pixel data representing pixels comprising a 
selected spatially distributed sub-set of pixels using a filter function selected 
from at least one of, (a) a vertical pixel data filter, (b) a horizontal pixel data 

30 filter, (c) a chrominance data filter, and (d) luminance data filter. 
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18. In a system for decoding compressed image data in the form of pixel 
blocks representing an image of a first resolution to provide an image of a reduced 
second resolution, a method comprising the steps of: 

generating data representative of an image pixel block at an intermediate 
5 third resolution lower than said first resolution but higher than said reduced second 
resolution; 

generating motion compensated pixel block data at said third resolution 
from pixel block data of said reduced second resolution supplemented by said 
intermediate third resolution data; and 
10 deriving pixel data representing said image of said reduced second 

resolution from said motion compensated pixel block data at said third resolution. 



15 



19. A method according to claim 18 wherein the steps of claim 
18 are performed for P frames exclusively of I and B frames. 

20. A method according to claim 18 wherein the steps of claim 
18 are performed for P frames and one of, (a) I frames and (b) B frames. 

21. A method according to claim 18 including the step of 

20 upsampling said pixel block data at said third resolution to 

provide image data of said first resolution. 

22. A method according to claim 21 including the step of 
downsampling said upsampled pixel block data of said first 

25 resolution to provide image data of said second resolution. 

22. A method according to claim 21 including the step of 
downsampling said upsampled pixel block data of said first 
resolution to provide said intermediate third resolution data. 



30 



23. A method according to claim 18 wherein 

said pixel block data of said third resolution comprises residual 

data. 
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ABSTRACT 

An improved image processing system involves decoding compressed image 
data including frequency domain coefficients defining blocks of pixel values 
representing an image at a first resolution to provide an image at a reduced second 
resolution for display from a selected sub-set of the frequency domain coefficients. The 
apparatus includes an enhanced motion-compensation-unit (MCU) operating with 
blocks of pixel values representing an image at an intermediate third resolution lower 
than the first resolution but higher than the reduced second resolution. 
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